Presentation: https://disk.yandex.com/i/ddOqenPSnV5DZg¶
Project "Telecom: Identifying Ineffective Operators":¶
The virtual telephony service CallMeMaybe is developing a new function that will give supervisors information on the least effective operators. An operator is considered ineffective if they have a large number of missed incoming calls (internal and external) and a long waiting time for incoming calls. Moreover, if an operator is supposed to make outgoing calls, a small number of them is also a sign of ineffectiveness.
In this research we are going study the available data on calls made by the operators in order to find which operators may be considered effective and non-effective ones. We'll also perform statistical significance tests in order to find if there's statistically signifacnt difference between two groups in terms of number of all calls made per day be one operator and in terms of share of internal calls to all calls made by an operator.
Step 1: Reading the dataset and looking at the general information¶
# importing libraries
import pandas as pd
from pandas import DataFrame
import numpy as np
! pip install missingno -U -q
import missingno as msno
%matplotlib inline
import matplotlib.pyplot as plt
import matplotlib as mpl
import seaborn as sns
! pip install plotly -U -q
import plotly.express as px
import plotly.figure_factory as ff
from plotly import graph_objects as go
import re
import math as mth
from scipy import stats as st
from scipy.stats import mannwhitneyu
from IPython.display import display
import sys
import warnings
if not sys.warnoptions:
warnings.simplefilter("ignore")
from datetime import datetime
# importing dataset
try:
calls = pd.read_csv('/Users/pavellugovoy/Desktop/data_analysis/final_project/main_project/telecom_dataset_us.csv')
clients = pd.read_csv('/Users/pavellugovoy/Desktop/data_analysis/final_project/main_project/telecom_clients_us.csv')
except:
calls = pd.read_csv('/datasets/telecom_dataset_us.csv')
clients = pd.read_csv('/datasets/telecom_clients_us.csv')
# looking at the general information of 'calls' dataset
calls.info()
display(calls.head(10))
display(calls.sample(10))
# looking at the metrics which can be generally evaluated with describe() method
calls.describe()
# looking at the general information of 'clients' dataset
clients.info()
display(clients.head(10))
display(clients.sample(10))
Conclusions¶
We have taken a first look on the datasets we have in hands and we see that the dataset on operators' activity (we named it 'calls') contains the following information:
- 'user_id' — client account ID;
- 'date' — date the statistics were retrieved;
- 'direction' — call direction ('out' for outgoing, 'in' for incoming);
- 'internal' — whether the call was internal (between a client's operators);
- 'operator_id' — operator identifier;
- 'is_missed_call' — whether the call was missed;
- 'calls_count' — number of calls;
- 'call_duration' — call duration (excluding waiting time);
- 'total_call_duration' — call duration (including waiting time).
The other dataset on the service's clients (we named it 'clients') contains the data on:
- 'user_id';
- 'tariff_plan' — client's current plan;
- 'date_start' — client's registration date.
We also have seen that the datasets have some problems:
- there are missing values in the columns of the table 'calls' containing information on whether the call was internal or not ('internal') and on operator's id ('operator_id').
- some of the data are of incorrect data type: the values which represent dates are of 'object' type, which is obviously wrong. Moreover there're the values of 'operator_id' in the table 'calls' look like integer numbers, but of float type.
We move further to the data prprocessing where we will check the missing values, change the inappropriate data types and check the data for duplicates.
Step 2: Prepocessing the data¶
2.1. Checking the data for missing values (including "hidden" ones) and preprocessing them.¶
# finding the number and percentage of missing values in 'calls'
report = calls.isna().sum().to_frame()
report = report.rename(columns = {0: 'missing_values'})
report['% of total'] = (report['missing_values'] / calls.shape[0]).round(2)
report.sort_values(by = 'missing_values', ascending = False)
# visualizing missing values with missingno library tool
msno.bar(calls)
plt.title ("Visualisation of missing values and non-missing values (*non-missing values are dark)", fontsize = 20)
plt.suptitle("")
plt.show()
Missing values in the column 'operator_id' of the table 'calls' make 15% of all values. It's quite a lot, but we have to remove the rows with these missing values: we cannot fill them, because these values are so called "missing at random", at the same time almost all our further analysis will focus on the operators' individual performance, therefore this data is mandatory and we will not be able to carry out the analysis without it,
AS for missing values in the column 'internal' of the table 'calls' they are also "missing at random", but the good news is that their number is not significant. So we can also remove them without a risk to lose a lot of information.
# removing rows with missing values
calls.dropna(inplace=True)
# checking the results
calls.info()
Now we'are going to check all other values for "hidden" missing values
checking 'user_id'¶
display(calls['user_id'].unique())
display(calls['user_id'].value_counts())
They look normal
checking 'date'¶
display(calls['date'].unique())
display(calls['date'].value_counts())
They look also fine
checking 'direction'¶
display(calls['is_missed_call'].unique())
display(calls['is_missed_call'].value_counts())
We see no problem here
checking 'is_missed_call'¶
display(calls['is_missed_call'].unique())
display(calls['is_missed_call'].value_counts())
No problem
checking 'calls_count'¶
display(calls['calls_count'].describe())
There's some problem as well. It is not possible to make 4817 calls a day. Probably, there was also a technical issue on extacting data step. Some calls were calculated multiple times. Let's see how the data distributed more precisely.
fig = px.histogram(calls, x="calls_count",
marginal="box",
hover_data=calls.columns)
fig.update_layout(
title = 'Distribution of number of calls per day" ',
xaxis_title = "Number of calls",
yaxis_title = "Number of entries in the dataset",
font=dict(
family="Arial",
size=12,
color="RebeccaPurple"
)
)
fig.show()
We see that "after" 120 calls there are a not so many entries in the bins. We may agree, that it is possible to make 120 calls per day. More than that it is highly impossible, so to be on a safe side we need to remove such data because it is very probable that this data is crewed.
checking 'call_duration'¶
display(calls['call_duration'].describe())
# plotting a histogram to see distribution of such values
fig = px.histogram(calls, x="call_duration",
marginal="box",
hover_data=calls.columns)
fig.update_layout(
title = 'Distribution of values related to "call_duration" ',
xaxis_title = "call duration",
yaxis_title = "Number of entries in the dataset",
font=dict(
family="Arial",
size=12,
color="RebeccaPurple"
)
)
fig.show()
checking 'total_call_duration'¶
display(calls['total_call_duration'].describe())
# plotting a histogram to see distribution of such values
fig = px.histogram(calls, x="total_call_duration",
marginal="box",
hover_data=calls.columns)
fig.update_layout(
title = 'Distribution of values related to "total_call_duration" ',
xaxis_title = "total call duration",
yaxis_title = "Number of entries in the dataset",
font=dict(
family="Arial",
size=12,
color="RebeccaPurple"
)
)
fig.show()
# looking at the rows with extra long duration
calls.query('total_call_duration > 28800').head(10)
It looks like there's a problem with these values. There are many outliers which are quite wierd. Something went wrong with the data, when it was extracted.
A normal working day (in the most of the countries) is 8 hours. 8 hours is 28 800 in total. So it is the maximum time that an operator may spend on calls. Of course, some operators may work extra hours, but they also need some rest and there's always some kind of waste of time.
So we assume that 8 hours period seems to be an appropriate time limit we may use as the fence in order to keep only most probable true data. Because the other data are also wrong (for example number of calls) the most reasonable way to handle such data is just to get rid of such values.
# filtering the data
calls = calls.query('call_duration <= 28800')
calls = calls.query('total_call_duration <= 28800')
calls = calls.query('calls_count < 120')
calls = calls.reset_index(drop=True)
# checking the results
calls.info()
2.2. Changing the inappropriate datatypes.¶
calls.head()
Converting data type of values of the column 'date' in the table 'calls' from object into datetime¶
# converting 'object' into 'datetime'
calls['date'] = pd.to_datetime(calls['date'])
# retreiving date from the each value
calls['date'] = calls['date'].dt.date
# convertung "back" into datetime
calls['date'] = pd.to_datetime(calls['date'])
Converting values in 'internal' column in the table 'calls' from 'object' into 'boolean' format¶
For sake of saving memory and improving processing speed we convert also values of 'internal' column to boolean column as they are actually of boolean type.
calls['internal'] = calls['internal'].convert_dtypes(convert_boolean=True)
Converting values in 'operator_id' column in the table 'calls' from 'float' into 'integer' format¶
calls['operator_id'] = calls['operator_id'].astype('int')
# checking the results
display(calls.head())
calls.info()
display(clients.head())
clients.info()
Converting data type of values of the column 'date_start' in the table 'clients' from object into datetime¶
# converting 'object' into 'datetime'
clients['date_start'] = pd.to_datetime(clients['date_start'])
display(clients.head())
clients.info()
2.3. Checking the dataset for duplicates.¶
Checking 'calls'¶
calls.duplicated().sum()
We found duplicates, so we need to drop them
calls = calls.drop_duplicates().reset_index(drop=True)
# checking the results
print(calls.duplicated().sum())
calls.info()
Checking 'clients'¶
clients.duplicated().sum()
No duplicates, so there's no more problem.
Conclusions¶
We have checked all the columns of the two datasets for missing values and have found that there are missing values related to the operators ids, such vales make about 15% of all data. less imporant but of the same kind We do not know the reason why these values are missing, but it looks like that there was a technical issue. But anyway, due the fact that we need them to evaluate efficiency of the operators (the main goal of the study), we had to removed them from the dataset.
Moreover we found that there were probable screwed data in the column 'calls_count', 'call_duration' and 'total_call_duration", so we removed the strange data as well.
We have found no other issues with missing values. We converted the data types to appropriate ones (dates to DateTime, integer to integers).We have found also that there were duplicates in the dataset 'calls', that we have also removed from the dataset.
Step 3: Carrying out EDA¶
3.1. Studying how often operators missed calls.¶
- Finding the ratio of missed calls to all calls received by each operator
calls_by_operator = calls.query('direction == "in"').pivot_table(index = 'operator_id', columns='is_missed_call', values ='direction', aggfunc ='count')
calls_by_operator.head()
# the reviewer's code:
calls_by_operator = ((calls
.query('direction == "in"')
.groupby(['operator_id'])
.agg(total_calls=('calls_count', 'sum'),
days=('date', 'nunique'))
.reset_index()
.merge((calls.query('direction == "in" & is_missed_call == True')
.groupby(['operator_id'])
.agg(mised_calls=('calls_count', 'sum')))
.reset_index(), on = "operator_id", how = 'left'))
.fillna(0).sort_values(by = 'mised_calls', ascending = False)
)
calls_by_operator.head()
# calculating the share of missed calls to all calls
calls_by_operator['share_missed'] = calls_by_operator['mised_calls']/calls_by_operator['total_calls']
# checking the results
calls_by_operator.head()
- Plotting a histogram showing the distribution of the said ratios, finding trends and outliers
fig = px.histogram(calls_by_operator, x="share_missed",
marginal="box", # or violin, rug
hover_data=calls_by_operator.columns)
fig.update_layout(
title = 'Distribution of values related to share of missed calls to all incoming calls ',
xaxis_title = "Share of missed calls to all incoming calls",
yaxis_title = "Number of operators",
font=dict(
family="Arial",
size=12,
color="RebeccaPurple"
)
)
fig.show()
We see on the histogram that there's a lot of operators with a relevant low share of missed calls, the upper fence for outliers according the "boxplot" method is 1,7%. But the number of outliers is quite high. So let's look what the said upper limit and and a little bit higher limit mean in terms of share of such operators to all operators.
len(calls_by_operator.query('share_missed >= 0.017'))/len(calls_by_operator)
17% of operators have a share of missed calls exceeding 1.7% limit. It's quite a lot. Let's look at the distribution of such values more precisely.
- Plotting density histograms showing the distribution of the said ratios
sns.distplot(calls_by_operator['share_missed'], hist = False, kde = True,
kde_kws = {'linewidth': 3},
label = 'share_missed')
plt.title ("Density plot for distribution of shares of missed calls to all calls", fontsize = 12)
plt.xlabel("Share of missed calls to all calls")
plt.ylabel("Density")
plt.suptitle("")
plt.show()
plt.show()
We see that the density (probability) of share of missed calls to all calls drastically falls around 8-10 %. Let's see what that mean in terms of share of operators and find mean, quartiles and the upper percentiles.
- Finding the mean, the median and quartiles of ratio values
calls_by_operator['share_missed'].describe()
# finding the 95's percentile
np.percentile(calls_by_operator['share_missed'], 95)
# finding 90's percentile
np.percentile(calls_by_operator['share_missed'], 90)
# finding 85's percentile
np.percentile(calls_by_operator['share_missed'], 85)
# finding 80's percentile
np.percentile(calls_by_operator['share_missed'], 80)
- Determing the threshold of the "high" ratio of missed calls to all calls received by an operator
It's hard to define after which rate the inefficiency commences in this case: there are a lot of "outliers", but their huge number means that they may be also "normal". i.e. they are maybe not outliers, that's just a trend. When we look at the dimensions of shares dividing operators into percentiles we see that the th share of missed calls for each percentile after 85's is growing drastically and 85's percentile threshold differs from 80's less than 90's from 85'. So it seems that 85' percentile is a good threshold for real outliers no matter which approach we use. That's why we decide to choose it as the threshold of efficiency performance.
# setting up a threshold as variable for future needs
missed_calls_thrld = np.percentile(calls_by_operator['share_missed'], 85)
3.2. Studying distribution of waiting time values for incoming calls.¶
- Filter the data by selecting only incoming calls and finding waiting time for each call and storing this data into a separate column
# creating a subset with only incoming calls and non-missed calls
incoming_calls = calls.query('direction == "in" and is_missed_call == False')
incoming_calls = incoming_calls.reset_index(drop=True)
incoming_calls.head()
# calculating waiting time for calls made on each date by each operator
incoming_calls['wait_time'] = incoming_calls['total_call_duration'] - incoming_calls['call_duration']
#incoming_calls['wait_time_per_call'] = incoming_calls['wait_time_all_calls']/incoming_calls['calls_count']
# checking the results
display(incoming_calls.sample(10))
display(incoming_calls.info())
- Finding average waiting time for all calls received by each operator
# creating a subset where we store the data on average waiting time for each operator
calls_by_operator_wait_time = (incoming_calls
.groupby('operator_id')
.agg(total_calls=('calls_count','sum'),
total_wait_time =('wait_time', 'sum'))
.reset_index()
)
calls_by_operator_wait_time['avg_per_call'] = (calls_by_operator_wait_time['total_wait_time']
/calls_by_operator_wait_time['total_calls'])
# checking the results
calls_by_operator_wait_time.sample(10)
- Plotting a histogram of waiting time values for calls received by operators
fig = px.histogram(calls_by_operator_wait_time, x="avg_per_call",
marginal="box", # or violin, rug
hover_data=calls_by_operator_wait_time.columns)
fig.update_layout(
title = 'Distribution of values related to average waiting time ',
xaxis_title = "Average waiting time",
yaxis_title = "Number of operators",
font=dict(
family="Arial",
size=12,
color="RebeccaPurple"
)
)
fig.show()
The upper-fence for identifying the outliers is 39,6 seconds. When we see at the histogram it seems that it's just: there are outliers, but they are clearly out of the trend. But let's at the share of such operators
len(calls_by_operator_wait_time.query('avg_per_call > 39.6'))/len(calls_by_operator_wait_time)
It is only 6 per cent of operators. So the most of the operators are "close" to each other in terms of average waiting time.
- Determing the threshold for long waiting time calls
It seems that the upper-fence for outliers according box-plot method is a good way to determine where inefficiency begins. The outliers defined by this approach are definitely out of the trend and less efficient than others. So we fix the threshold accordingly.
# setting up the threshold for long waiting time calls
long_wait_time__thrld = 39.6
3.3. Studying distribution of numbers of outgoing calls.¶
- Finding average number of calls made per day by an operator
calls.head()
calls.info()
We need first define if a given operator is supposed to make outgoing calls.
Let's filter the data by 'internal' field (the calls shall be external) and by the direction field (the calls shall be outgoing).
# creating a subset with filtered the data by 'direction' field and 'internal' field
outgoing_calls = (calls[(calls["internal"] == False) & (calls["direction"] == "out")]).reset_index(drop=True)
# checking the results
display(outgoing_calls.head())
outgoing_calls.info()
# grouping data by operator and by date and calculating the number of calls
outgoing_calls_operator = outgoing_calls.groupby('operator_id').agg(calls_avg_n=('calls_count','mean'))
# checking the results
outgoing_calls_operator.head(20)
- Plotting a histrogram showing distribution of average number of outgoing calls through operators, finding trends and outliers
fig = px.histogram(outgoing_calls_operator, x="calls_avg_n",
marginal="box",
hover_data=outgoing_calls_operator.columns)
fig.update_layout(
title = 'Distribution of average numbers of outgoing calls per day and per operator',
xaxis_title = "Average number of outgoing calls per day and per operator",
yaxis_title = "Number of operators",
font=dict(
family="Arial",
size=12,
color="RebeccaPurple"
)
)
fig.show()
outgoing_calls_operator["calls_avg_n"].describe()
The picture is intersting. There are a lot of operators who made a really small number of calls. It's not very probable that all of them are inefficient. Probably they are just not supposed to make such calls. Perhabs they make them by occasion with incoming calls. In that case we cannot say they are ineffective.
Let's see if the operators making outgoing calls receive also incoming calls, and if so, then how many such calls they receive compare making outgoing calls.
# creating a subset where we calculate incoming calls and outgoing calls
outgoing_callers= ((calls.query('internal == False & direction == "out"')
.groupby('operator_id')
.agg(total_calls_out=('calls_count','sum'),
avg_calls_out=('calls_count', 'mean'))
.reset_index())
.merge((calls.query('internal == False & direction == "in"')
.groupby('operator_id')
.agg(total_calls_in=('calls_count','sum'),
avg_calls_in=('calls_count', 'mean'))
.reset_index()), on='operator_id', how='left')
.fillna(0)
)
# checking the results
outgoing_callers.head(10)
Let's see how many received no incoming calls
len(outgoing_callers.query('total_calls_in == 0'))
Not a lot, let's which operators made more outgoing calls than incoming
# finding a ratio of average number of incoming calls to outgoing calls
outgoing_callers['ratio_avg_in_out'] = (outgoing_callers['avg_calls_in']
/outgoing_callers['avg_calls_out'])
# checking the results
outgoing_callers.sample(10)
outgoing_callers['ratio_avg_in_out'].describe()
len(outgoing_callers.query('ratio_avg_in_out <= 1'))
More operators. They are definitely supposed to make outgoing calls - they make them more than incoming. So we filter the subset for such operators and then find the threshold of efficiency.
outgoing_callers = outgoing_callers.query('ratio_avg_in_out <= 1')
outgoing_callers = outgoing_callers.reset_index(drop=True)
# checking the results
outgoing_callers.sample(10)
- Looking at the distribution of average number of outgoing calls once again
fig = px.histogram(outgoing_callers, x="avg_calls_out",
marginal="box",
hover_data=outgoing_callers.columns)
fig.update_layout(
title = 'Distribution of average numbers of outgoing calls per day and per operator',
xaxis_title = "Average number of outgoing calls per day and per operator",
yaxis_title = "Number of operators",
font=dict(
family="Arial",
size=12,
color="RebeccaPurple"
)
)
fig.show()
outgoing_callers['avg_calls_out'].describe()
# finding 40's percentile
np.percentile(outgoing_callers['avg_calls_out'], 40)
In fact, even after we filtered the data using a quite sofisticated way the picture is not much changed, we see that a great part of operators make really small number of calls. But maybe it's really the case: there is also a good part of operators which make a relatively higher numbe of calls, they seem to be effective and the others are not.
- Determing threshold for the low number of calls
Taking into account the relevant efficient part operators which made more than 10 calls per day, we find apporpriate to set up the relevant threshold as 6 calls per day
# setting up the threshold for number of outgoing calls
outgoing_calls_thrld = 6
Step 4: Identifying Ineffective Operators¶
- Classifying the operators as effective and ineffective based on the thresholds for the metrics found during the EDA
# creating a dataframe with operator id of all operators
operators_series = pd.Series(calls['operator_id'].unique())
operators = pd.DataFrame(operators_series)
operators = operators.rename(columns={0:'operator_id'})
# checking the results
display(operators.head())
display(operators.tail())
operators.info()
# creating a list of ineffective operators based on the threshold for ratio of missed calls
missed_calls_list_ineffective = (calls_by_operator.query('share_missed > @missed_calls_thrld')['operator_id'].unique())
# creating a list of ineffective operators based on the threshold for ratio of missed calls
long_wait_time_list_ineffective = (calls_by_operator_wait_time
.query('avg_per_call > @long_wait_time__thrld')['operator_id'].unique())
# creating a list of ineffective operators based on the thereshold for average number of outgoing calls
small_number_outgoing_calls_ineffective = (outgoing_callers
.query('avg_calls_out < @outgoing_calls_thrld')['operator_id'].unique())
let's find the length of each list
len(missed_calls_list_ineffective)
len(long_wait_time_list_ineffective)
len(small_number_outgoing_calls_ineffective)
# creating a function returning a label for operator id
def efficiency_classifier(operator_id):
if (operator_id in missed_calls_list_ineffective
and operator_id in long_wait_time_list_ineffective
or operator_id in small_number_outgoing_calls_ineffective):
return 'ineffective'
else:
return 'effective'
# testing if the function works
efficiency_classifier(958394)
# applaying the function to operators ids and getting the value indicating if they are effective or not
operators['efficiency']=operators['operator_id'].apply(lambda x: efficiency_classifier(x))
operators.head()
# looking at the split of the operators into two groups
operators['efficiency'].value_counts()
Let's look at the ratios of two groups of operators two all operators
# calculating the ratio of effective operators to all operators
effective_operators_rat = len(operators[operators['efficiency'] == 'effective'])/len(operators)
# calculating the ratio of effective operators to all operators
1 - effective_operators_rat
We have finally got a table of effective and ineffective operators. According to our approach the ineffictive operators make around 30 % of all operators. It's quite high. But the most part of them are operators making outgling calls. That is the issue.
Step 5: Statistical hypotheses testing¶
5.1. Testing hypothesis that the average number of all calls made or received per day by an operator differs within effective and ineffective operators.¶
First, we find the average number of calls per day for each operator and then we will use our function once again in order to split the list into two groups.
calls.head()
# calculating the number of calls for each day, for each operator
all_calls_by_operator = calls.query('is_missed_call == False').groupby(['operator_id', 'date']).agg({'calls_count':'sum'}).reset_index()
all_calls_by_operator = all_calls_by_operator.groupby('operator_id').agg(n_calls=('calls_count', 'mean'))
all_calls_by_operator = all_calls_by_operator.reset_index()
# checking the results
display(all_calls_by_operator.sample(10))
all_calls_by_operator.info()
# applying the written function to classify the operators
all_calls_by_operator['efficiency'] = (all_calls_by_operator['operator_id']
.apply(lambda x: efficiency_classifier(x))
)
# checking the results
all_calls_by_operator.head()
Now let's see if the average number of calls differs between the two groups
all_calls_efficiency = all_calls_by_operator.groupby('efficiency').agg({'n_calls':'mean'})
all_calls_efficiency
The mean values for two groups differ significantly, but let's see how the average number of calls values are distributed within two groups by looking at a histogram
fig = px.histogram(all_calls_by_operator, x="n_calls", color='efficiency',
marginal="box",
hover_data=all_calls_by_operator.columns)
fig.update_layout(
title = 'Average numbers of calls per day within effective and ineffective operators',
xaxis_title = "Average number of calls per day",
yaxis_title = "Number of operators",
font=dict(
family="Arial",
size=12,
color="RebeccaPurple"
)
)
fig.show()
We see that there is a difference in disribution of the values.
Let's test if this difference is significant using Mann-Whitney U Test. We formulate the null hypothesis (H0) as "there's no difference in distributions of values" and the alternate hypothesis (H1) as "there's a difference in distributions of values". We set up the significance level as "0.05" which is sufficient in our case as there's no multicollinearity in performing this test (it is actually the only one).
# creating two samples
effective_operators_calls_sample = (
all_calls_by_operator[all_calls_by_operator['efficiency'] =='effective']['n_calls']
)
ineffective_operators_calls_sample = (
all_calls_by_operator[all_calls_by_operator['efficiency'] =='ineffective']['n_calls']
)
# comparing samples
stat, p = mannwhitneyu(effective_operators_calls_sample, ineffective_operators_calls_sample)
print('Statistics=%.3f, p=%.5f' % (stat, p))
# interpretating
alpha = 0.05
if p > alpha:
print('Same distribution (fail to reject H0)')
else:
print('Different distribution (reject H0)')
We see that the distribution is not "equal". So we may say, that effective users make on everage more calls and even if it was not a part of the metrics to measure efficiency, we see that there's a difference. At the same time we need to notice that the main part of the ineffective operators is made by those who made not a "large" number of outgoing calls, so the result of such a hypothesis may be screwed by this fact.
5.2. Testing hypothesis that the share of internal calls to all calls differs within effective and ineffective operators.¶
calls.head()
# creating a subset with data on number of external and internal calls split by operator
internal_calls_by_operators = calls.pivot_table(columns='internal', index='operator_id', aggfunc='sum', values='calls_count')
internal_calls_by_operators.reset_index(inplace=True)
internal_calls_by_operators = (internal_calls_by_operators
.rename_axis(None, axis=1).reset_index(drop=True)
)
internal_calls_by_operators = (internal_calls_by_operators
.rename(columns={False:'external', True:'internal'})
)
internal_calls_by_operators['all_calls'] = (internal_calls_by_operators['external']
+ internal_calls_by_operators['internal']
)
internal_calls_by_operators['internal_share'] = (internal_calls_by_operators['internal']
/internal_calls_by_operators['all_calls'])
# applying the written function to classify the operators
internal_calls_by_operators['efficiency'] = (internal_calls_by_operators['operator_id']
.apply(lambda x: efficiency_classifier(x) )
)
internal_calls_by_operators.head()
Now let's look at mean values of shares of internal calls within two groups
internal_calls_by_operators.groupby('efficiency').agg({'internal_share':'mean'})
We see that the two groups differ from each other quite significant in terms of mean share of internal calls to all calls. Let's see more precisely at this situation by looking as usual at the histogram and the boxplot descring the distribution between two samples.
fig = px.histogram(internal_calls_by_operators, x="internal_share", color='efficiency',
marginal="box",
hover_data=internal_calls_by_operators.columns)
fig.update_layout(
title = 'Distribution of shares of internal calls to all calls made by an operator',
xaxis_title = "Share of internal calls to all calls made by an operator",
yaxis_title = "Number of operators",
font=dict(
family="Arial",
size=12,
color="RebeccaPurple"
)
)
fig.show()
We see that the distribution of values within two samples differs. But we need to find if such differenct is statistically significant. As above, we'll use Mann-Whitney U Test. And again, as before we formulate the null hypothesis (H0) as "there's no difference in distributions of values" and the alternate hypothesis (H1) as "there's a difference in distributions of values".
We set up the significance level as "0.05" which is sufficient in this case as well as there's no multicollinearity in performing with other tests.
# creating two samples
effective_operators_internal_calls_sample = (
internal_calls_by_operators[internal_calls_by_operators['efficiency'] =='effective']['internal_share']
)
ineffective_operators_internal_calls_sample = (
internal_calls_by_operators[internal_calls_by_operators['efficiency'] =='ineffective']['internal_share']
)
# comparing samples
stat, p = mannwhitneyu(effective_operators_internal_calls_sample, ineffective_operators_internal_calls_sample)
print('Statistics=%.3f, p=%.5f' % (stat, p))
# interpretating
alpha = 0.05
if p > alpha:
print('Same distribution (fail to reject H0)')
else:
print('Different distribution (reject H0)')
As we see the test shows that we have no ground to accept the null hypothesis, therefore we may say that the difference between the distributions of two samples is statistically significant.
It's a quite interesting fact. It means that "effective" operators made less internal calls. We do not know why exactly operators had to make internal calls. It may be explained by the fact that operators make them in order to put questions to their colleagues because they lack of expertise and/or the scripts they use are not clear. If so, it is what we need to pay attention to, because it may be a key to improving the efficiency of operators performance.
Step 6: General conlusions and recommendations¶
We have made a retrospective analysis of operators performance in order to find which thresholds we may use in order to understand which operators are ineffective in terms of the following metrics:
- long waiting time for incoming calls,
- share of missed calls to all incoming calls,
- small number of outgoing calls
We have explored the data we have in hands and we have determined the thresholds which were:
- in the case of share of long waiting time 39,6 seconds,
- in the case of share of missed calls we defined that this share as the dimension of such share dividing 85 % operators from 15% less effective.
- the most arguable question is the threshold for the number of outgoing calls. First, it was not very easy to understand which operators were supposed to make outgoing calls and which were not. Finally we decided to take into account only those who made per day more external outgoing calls than receiving externl incoming calls. But anyway, we found that the was a really big part of operators which made less 6 outgoing calls per day, while there are other, who made much more. Finally, we set up such threshold accordingly.
Not a suprise that the latter metric gives the main number of ineffective operators. So we suggest to evaluate once again if it is worth using such a metric in our situation. Perhaps it is more appropriate to use some other (such as the share of made calls to then number of calls defined by a specific KPI).
Taking into account the said assumptions we compared two groups of operators by testing two hypotheses:
- the number of calls made per day by an operators differs within two groups,
- the share of internal calls to all calls difers within two groups.
We first calculated the said metrics and then performed tests of significance of differences of distributions within two samples.
The results were quite intersting: we found that the operators do differ by the number of all calls and they differ in terms of share of internal calls to all calls.
The first outcome is not very reliable due to the fact that the a great part of the ineffective operators is made by those who make not enough ougoing calls.
But the latter fact may be a sign or a key to improving operators performance. We do not know why operators make internal calls, but perhaps they make them because they lack of self-confidence and/or scripts are not very good, therefore they need to call their colleagues/receive calls from colleagues inspite of making calls/receiving incoming calls.
List of sources¶
- https://en.wikipedia.org/wiki/Automatic_call_distributor - a Wikipedia article giving a brief information on what automatic call distribution is,
- https://towardsdatascience.com/visualize-missing-values-with-missingno-ad4d938b00a1 - an article on missingno library,
- https://stackoverflow.com/questions/16176996/keep-only-date-part-when-using-pandas-to-datetime - how to retrieve date from datetime keeping the datetime format,
- https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.convert_dtypes.html - how to convert different types of data format,
- https://plotly.com/python/distplot/ - plotly library distribution plots manual,
- https://www.geeksforgeeks.org/creating-a-dataframe-from-pandas-series/ - article on how to create a Pandas DataFrame from a series object,
- https://searchcustomerexperience.techtarget.com/tip/8-customer-service-metrics-to-measure-call-center-success - article with regard to the some techniques to measure productivity of call center agents(operators)